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Arkansas has identified college and career readiness indicators for schools that can be used to monitor 
students’ performance and to improve their postsecondary readiness and success. Using two cohorts of grade 
6 students, this study examined the extent to which Arkansas’s middle school and high school indicators of 
postsecondary readiness predict a student postsecondary readiness outcome (an ACT score of 19 or higher) 
and success outcomes (enrolled in college for at least one term within eight years of beginning grade 6, 


and persisted in college by enrolling for more than one term within eight years of beginning grade 6). The 
study estimated the accuracy and strength of the middle school and high school indicators for predicting 
the outcomes. While fewer than half of students met the Arkansas postsecondary readiness standard, more 
than half enrolled in college and about half persisted for more than one term within eight years of beginning 
ig-Le(-M om Yee |(-mxelatore) W-laveMalt=4amexel aero) Mlavel(or-1colemmUYatslamxelaalelinsomiaivamid(e(-\aim e-la¢-4celUlileKaar-le-vak-le cy aloy 
predicted readiness and success outcomes with greater accuracy than did student background characteristics 
alone. Middle school indicators that were major predictors for at least two of the three outcomes examined 
included proficiency in English language arts and math, regular school attendance, no suspensions, and no 
expulsions. High school indictors that were major predictors for at least two of the outcomes included grade 
point average, enrollment in an advanced course, regular school attendance, and no expulsions. 


State and local education agencies throughout the country have established goals for students’ postsecondary 
readiness and success. The Arkansas Department of Education has established the goal that all students will grad- 
uate from high school prepared for college, career, and community engagement. Arkansas faces several challeng- 
es as it seeks to achieve this ambitious goal. 


Researchers estimate that more than a third of students nationally have left high school without the requisite 
skills to succeed in college or career (Carnevale et al., 2013; Cataldi & KewalRamani, 2009; Conley, 2012; Sparks 
& Malkus, 2013). Readiness for college and careers is lower among students of color and students with disabil- 
ities than among their peers (Aud et al., 2013; Greene & Winters, 2006). Evidence suggests that postsecondary 
readiness in Arkansas lags national levels. For example, the percentage of 2018 ACT-tested high school graduates 
meeting ACT college readiness benchmarks was lower in Arkansas than in the United States overall in each of four 
tested subjects—English, reading, math, and science (ACT, 2018). 


To address these challenges systematically and rigorously, researchers and policymakers have developed strat- 
egies to track students’ progress and identify data-driven indicators that can predict their postsecondary readi- 
ness and success (Bowers et al., 2012; Dynarski et al., 2008; Stephan et al., 2015). Longitudinal data systems and 
early warning indicators are intended to track and identify malleable aspects of 
student engagement and learning that can enhance later life outcomes (Cataldi For additional information, 
& KewalRamani, 2009; Hartman et al., 2011; Heppen & Therriault, 2008; Stephan including technical 

et al., 2015). To this end the Arkansas Department of Education has invested in methods, supporting 
postsecondary tracking systems, and the Arkansas Every Student Succeeds Act 
plan (Arkansas ESSA plan) includes student-level School Quality and Student 
Success indicators for which data are available (Arkansas Department of Educa- 
tion, 2017). 


tables, and alternative 
model results, access 

the report appendixes at 
https://go.usa.gov/xHukK. 
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To advance its college, career, and community engagement preparation goals, the Arkansas Department of Educa- 
tion and its Arkansas Division of Higher Education requested that the Regional Educational Laboratory Southwest 
conduct a study to measure the number of students attaining readiness and success outcomes and to identify 
which ESSA plan indicators in middle school and high school are good predictors of postsecondary readiness and 
success.' Knowing which indicators have strong predictive accuracy can help state and local education agencies 
identify students who are on track and those who are off track to attain postsecondary readiness and success. 
The results from this study identifying indicators that are good predictors of postsecondary readiness and success 
can guide timely and targeted student-, classroom-, and school-level interventions designed to enhance students’ 
chances of postsecondary readiness and success. The study’s findings can also guide state and local policymakers 
in directing resources toward students and schools in greatest need of support. 


Research questions 
This study addressed three questions: 


1. What percentage of Arkansas students attained the postsecondary readiness outcome (ACT score of 19 or 
higher) and success outcomes (college enrollment and persistence), and did attainment differ according to 
student background characteristics or status on postsecondary readiness indicators from middle school and 
high school? 


2. How accurately do postsecondary readiness indicators from middle school and high school predict attainment 
of the postsecondary readiness outcome (ACT score of 19 or higher) and success outcomes (college enrollment 
and persistence)? Does using the postsecondary readiness indicators improve the accuracy of these outcome 
predictors compared with using only student background characteristics? 


3. After student background characteristics are controlled for, which middle school and high school indicators 
are the strongest predictors of the postsecondary readiness outcome (ACT score of 19 or higher) and success 
outcomes (college enrollment and persistence)? 


Key terms used in this report are defined in box 1. The data sources, study population, methods, and limitations 
are summarized in box 2 (see appendix A for additional information). 


Box 1. Key terms 


High school indicators. Indicators of postsecondary readiness and success that measure whether students, when they were in 

high school: 

e Scored proficient or higher at least once on a state math assessment. 

e Scored proficient or higher on a state science assessment. 

e Attained a grade point average of 2.8 or higher. 

e Enrolled in at least one advanced course (Advanced Career Education, Advanced Placement, or International Baccalaureate). 

e Enrolled in at least one community service learning course. 

e Were present more than 95 percent of days enrolled for all years. 

e Were present more than 90 percent but not more than 95 percent (91-95 percent) of days enrolled in at least one year, and 
were not chronically absent (absent 10 percent or more of days) in any year. 

e Were never suspended. 

e Were never expelled. 


1. Pursuant to the Transformation and Efficiencies Act of 2019, the Arkansas Department of Higher Education was renamed the Arkansas 
Division of Higher Education, a division within the Arkansas Department of Education. 
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Major predictor. Defined for this study as an indicator that has a statistically significant correlation with an outcome and that 
predicts at least a 10 percentage point change in the probability of a student attaining an outcome (see box 2). 


Middle school indicators. Indicators of postsecondary readiness and success that measure whether students, when they were 
in middle school: 


e Scored proficient or higher in grade 8 on a state English language arts assessment. 

e Scored proficient or higher in grade 8 on a state math assessment. 

e Scored proficient or higher in grade 7 on the state science assessment. 

e Were present more than 95 percent of days enrolled for all years during grades 6-8. 

e Were present more than 90 percent but not more than 95 percent (91-95 percent) of days enrolled in at least one year, and 
were not chronically absent (absent 10 percent or more of days) in any year. 

e Were never suspended. 

e Were never expelled. 


Postsecondary readiness. Students have the knowledge, skills, and attributes that high school students are expected to have 
acquired or demonstrated to enable their successful entry into postsecondary education and employment. 


Postsecondary readiness indicators. Measures of student academic and behavioral experiences from middle school and high 
school that may predict or correlate with future college and career readiness and success. This study examined middle school and 
high school indicators that align with School Quality and Student Success indicators in the Arkansas Every Student Succeeds Act 
(ESSA) plan and for which data were available (see table A1 in appendix A). 


Postsecondary readiness outcome (ACT score). Defined for the study as a score of 19 or higher on the ACT exam (offered to 
all grade 11 public school students in Arkansas free of charge). That score is the ACT standard specified in the Arkansas ESSA plan. 


Postsecondary success outcomes (enrollment and persistence). This study examined two postsecondary success outcomes, 
which describe the attainment of education, training, or employment outcomes following high school: 


¢ College enrollment: enrolled for at least one term in a higher education institution, regardless of the degree or certificate 
being pursued or attained, within eight years of beginning grade 6 (that is, within two years of expected on-time high school 
graduation). 

e Postsecondary persistence: enrolled for more than one term in a higher education institution within eight years of beginning 
grade 6, including enrollment in nonconsecutive terms and enrollment in more than one institution. 


Predictive accuracy. Defined for the study as the percentage of students with the same predicted and actual outcomes (that is, 
true positives or true negatives), based on logistic regression models. True positives occur when a student who was predicted to 
attain readiness or success attained readiness or success. True negatives occur when a student who was predicted not to attain 
readiness or success did not attain readiness or success. False positives occur when a student who was predicted to attain readi- 
ness or success did not attain readiness or success. False negatives occur when a student who was predicted not to attain readi- 
ness or success attained readiness or success. 


Predictive strength. Defined for the study as the partial correlation between an individual indicator and an outcome measure 
within the context of a regression model that controls for other indicators and student background characteristics. 


Student background characteristics. The study examined seven background characteristics, all measured as of grade 6: 


e Gender. 

¢ Race/ethnicity. 

e Eligibility for the national school lunch program (an indicator of economic disadvantage). 

e Designation as an English learner student. 

¢ Disability designation. 

e Whether a student was older than typical age (age 13 or older) at the beginning of grade 6. 
¢ Geographic locale of district (urban, suburban, town, or rural). 
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Box 2. Data sources and study population and methods 


Data sources. The study used three primary types of grade 6-12 student-level data from the Arkansas Department of Education 
for school years 2008/09-2017/18 for students who were enrolled in grade 6 in Arkansas public schools in 2008/09 or 2009/10: 
student background characteristics, student ACT scores (an assessment of college readiness during high school), and student 
academic and behavioral experiences in middle school and high school.’ Student background characteristics and school district 
locale (urban, suburban, town, or rural) were based on grade 6 records. Data for student academic and behavioral experiences/ 
indicators in middle school and high school included Arkansas Comprehensive Testing, Assessment, and Accountability Program 
benchmark assessment scores for English language arts, math, and science; high school course transcript records; attendance 
records; and discipline records. Students whose grade 6-12 records indicated that they had left the school system for reasons 
other than graduation or dropping out were excluded from the analyses. 

The study used student-level college enrollment records from the Arkansas Division of Higher Education for public and private 
higher education institutions in Arkansas and from the National Student Clearinghouse (provided by the Arkansas Department of 
Education) for out-of-state higher education institutions in the United States. The Arkansas Department of Education provided 
unique random student identifiers to link data across sources and years. 


Population. The study examined a population of 63,679 students who were enrolled in grade 6 in Arkansas public schools in 2008/09 
or 2009/10 and tracked these students for eight additional years, until 2016/17 (the 2008/09 cohort) or 2017/18 (the 2009/10 cohort). 
Students who enrolled in later grades and years but were not enrolled in grade 6 in 2008/09 or 2009/10 were excluded from the 
analysis. White students were the largest racial/ethnic group in the population (65 percent), and a majority (60 percent) of the stu- 
dents in the population were eligible for the national school lunch program (an indicator of economic disadvantage) in grade 6 (see 
table A2 in appendix A). Students in rural districts represented the largest proportion by locale (39 percent). Appendix A provides 
additional details on which students were excluded from the analyses and on how cases with missing ACT scores were handled. 


Methods. The study team used academic and behavioral data to construct indicators of postsecondary readiness and success 
based on the indicators described in Arkansas’s Every Student Succeeds Act plan (see box 1). The study team used one set of 
middle school indicators and two approaches to construct high school indicators, guided by the timing of the outcome variable 
being predicted. Rather than constructing indicators for each grade level, the study team constructed indicators from combined 
data across middle school grades and combined data across high school grades (see appendix A for detail). For research question 1 
the study team completed a series of univariate and bivariate descriptive analyses and calculated the percentage of students who 
attained each of the three postsecondary readiness and success outcomes. 

The study team also calculated attainment of each outcome for groups of students by student background characteristics and 
by student status on the middle school and high school indicators examined in the study. The study team defined group differenc- 
es of at least 10 percentage points as major. In defining major differences in this way, the study team was attentive to magnitudes 
of difference that would be meaningful to state and local education agencies. 

For research questions 2 and 3 the study team constructed a set of logistic regression models that estimated the probability 
of students attaining the three outcomes of interest. Three models were tested, including models that had as predictors student 
background characteristics alone, models that included student background characteristics plus middle school indicators, and 
then models that included student background characteristics plus high school indicators. The results of models that included 
middle school and high school indicators simultaneously are not shown because of their multicollinearity (see appendix A for 
detail). To avoid concerns that the predictive findings apply only to the students whose data were used to fit the models and thus 
are not broadly applicable, a simple out-of-sample validation strategy was applied in all analyses: data for a randomly selected 
70 percent of the population was used to train, or fit, the data and then data for the remaining 30 percent of the population was 
used to test the accuracy of the model. The report presents findings based on this testing sample (see appendix B for detail). 

Research question 2 examined the accuracy of student background characteristics and middle school or high school indica- 
tors in predicting the outcomes by comparing students’ predicted attainment of an outcome with their actual attainment of the 
outcome. The predicted attainment criterion was met when a student’s predicted likelihood of attaining an outcome was at least 
50 percent (Hastie et al., 2009; see appendix A for detail). Based on this criterion, each student was assigned to one of four cate- 
gories (true positive, true negative, false positive, or false negative) that describe whether the model predicted that an outcome 
was attained or not attained and whether the predicted and actual outcomes matched (true or false; see table A4 in appendix A). 
The accuracy of the model was calculated as the number of true predictions divided by the total sample size. 
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Research question 3 examined the predictive strength of the middle school and high school indicators. The study team 
used results from the regression models to estimate percentage point changes in the predicted probability of an outcome being 
attained when an indicator’s value changed from zero to one (for example, from present in school fewer than 90 percent of days 
enrolled to present at least 90 percent of days enrolled), after other variables were controlled for. Because of the large population 
of students analyzed, the models could detect statistically significant relationships for relatively low correlations. Therefore, the 
study team defined as major predictors of outcomes indicators that met two criteria: statistically significant correlation with an 
outcome and predicting at least a 10 percentage point change in the probability of a student attaining an outcome. 

For research questions 2 and 3 the study team also used random forest models, a type of machine learning, as an alterna- 
tive to logistic regression models. Appendix C explains the reasons for exploring random forest findings and how these findings 
compare with the results of the logistic regression models. 


Limitations. The study data and methods have several limitations. The study used data from cohorts that entered the middle 
grades over a decade ago to estimate relationships between student indicators for grades 6-12 and postsecondary readiness and 
success outcomes. The contexts and relationships between indicators and future outcomes might not be similar for current stu- 
dents. And the study examined indicators from the Arkansas School Quality and Student Success indicators (see table A1 in appen- 
dix A) for which data were available in the administrative records shared with the study team. Data were unavailable on some 
other aspects of student experience and learning that prior research has shown to be associated with postsecondary outcomes. 
Additional limitations are presented in appendix A. 


Note 


1. Any student who was enrolled in grade 6 in both 2008/09 and 2009/10 (that is, repeated grade 6) was treated as a member of the 2008/09 cohort 
and appeared as only a single case in the analyses. 


Student attainment of postsecondary readiness and success outcomes 


This section presents descriptive findings for the percentages of students from the 2008/09 and 2009/10 grade 
6 cohorts who attained Arkansas’s readiness (ACT score) standard and success outcomes (college enrollment and 
persistence).? Appendix B provides supporting tables. 


About 39 percent of students met the Arkansas postsecondary readiness standard (ACT score of 19 or 
higher), but only 60 percent took the exam 


While 39 percent of all students met the Arkansas readiness standard by scoring 19 or higher on the ACT in high 
school, another 21 percent took the ACT in high school but did not meet the readiness standard (figure 1). Forty 
percent of students did not take the ACT while in high school.? 


More than half of students enrolled in college, and slightly less than half persisted in college 
Fifty-eight percent of students in the study enrolled in college within eight years of beginning grade 6 (figure 2; 


see table B3 in appendix B).* Forty-nine percent of students enrolled in college for more than one term within 
eight years of beginning grade 6. 


2. All results reported in the findings section are based on combining the two analytic cohorts. Cohort-specific findings were generally 
similar to the findings for the combined cohorts, except where noted in appendix B. 

3. The regression analyses addressing research questions 2 and 3 excluded students lacking ACT records. Appendix A summarizes a sensi- 
tivity analysis examining whether estimates changed if students lacking ACT records were included in the analysis. 

4. Records indicated that 19.7 percent of study students participated in concurrent high school and college (dual enrollment) courses. 
Consistent with the literature, which indicates that dual enrollment students tend to persist and graduate from college at higher 
rates than other students (Allen & Dadgar, 2012; An & Taylor, 2019), Arkansas Department of Education records showed that these 
concurrent enrolled students were more likely to enroll in college for multiple terms than other students in the sample (90 percent as 
compared with 41 percent). 
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Figure 1. About 60 percent of students from the 2008/09 and 2009/10 grade 6 cohorts took the ACT in high 
school, and a majority of that group met the Arkansas postsecondary readiness outcome (ACT score of 19 or 
higher), 2008/09-2017/18 


@ ACT score of 19 or higher m™ ACT score of less than19 ™ No ACT score 
Percent of students 


100 


80 


All students Students who took the ACT 


Note: n = 63,679. 


Source: Authors’ analysis of data for 2008/09-2017/18 from the Arkansas Department of Education, Arkansas Division of Higher Education, National 
Student Clearinghouse, and National Center for Education Statistics Common Core of Data (U.S. Department of Education, n.d.). 


Figure 2. About half of students from the 2008/09 and 2009/10 grade 6 cohorts attained the Arkansas 
postsecondary success outcomes (college enrollment and persistence) within eight years, 2008/09-2017/18 


Percent of students 
100 


Enrollment? Persistence? 


Note: n = 63,679. 
a. Enrolled in college for at least one term, regardless of the degree or certificate being pursued or attained, within eight years of beginning grade 6. 
b. Enrolled in college for more than one term within eight years of beginning grade 6. 


Source: Authors’ analysis of data for 2008/09-2017/18 from the Arkansas Department of Education, Arkansas Division of Higher Education, National 
Student Clearinghouse, and National Center for Education Statistics Common Core of Data (U.S. Department of Education, n.d.). 


Outcome attainment varied greatly by student groups. Attainment of the readiness (ACT score) and success out- 
comes (college enrollment and persistence) was lower for students with a disability designation, students eligible 
for the national school lunch program, students who were older than the typical age in grade 6, and English 
learner students than for students without these characteristics (see table B1 in appendix B). 


Black/African American students and Hispanic students had lower attainment of the readiness (ACT score) 
and success outcomes (college enrollment and persistence) relative to White students. Compared with female 
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students, male students had similar attainment of the readiness outcome but lower attainment of the success 
outcomes. 


For the readiness outcome (ACT score) students who were enrolled in an urban district in grade 6 had lower 
attainment than did students enrolled in a suburban district. For the success outcomes (college enrollment and 
persistence) students in urban districts had lower attainment than students in suburban, town, or rural districts. 


Outcomes attainment varied greatly by middle school and high school indicator status. Among the middle school 
indicators some of the largest differences in outcome attainment (32-63 percentage points higher) were for 
students who demonstrated proficiency in English language arts and math compared with students who did 
not demonstrate proficiency (see tables B2 and B3 in appendix B). Additionally, students who were proficient 
in science, were never expelled, were never suspended, and were not chronically absent> had 12-50 percent- 
age point higher attainment for each readiness or success outcome than their counterparts for each of these 
indicators. 


Among the high school indicators some of the largest differences in outcome attainment (31-56 percentage points 
higher) were for students who demonstrated proficiency in math and science compared with students who did 
not demonstrate proficiency, for students who earned a grade point average of 2.8 or higher compared with stu- 
dents with a lower grade point average, and for students who took at least one advanced course compared with 
students who did not (see tables B4 and B5 in appendix B). Additionally, students who were never suspended, 
were never expelled, and were not chronically absent had a 10—42 percentage point higher attainment for each 
readiness or success outcome compared with their counterparts for each of these indicators. 


Predictive accuracy of middle or high school indicators 


This section presents findings on changes in predictive accuracy for the three outcomes when middle school 
or high school indicators aligned with the Arkansas ESSA plan were included in the models along with student 
background characteristics compared with findings from models that included only student background charac- 
teristics. The study defined predictive accuracy according to the percentage of students with the same predicted 
and actual outcomes (that is, true positives or true negatives), based on logistic regression models that included 
middle school or high school indicators. (Appendix B provides supporting tables.°) The study also used random 
forest analyses, a type of machine learning model, to conduct a supplemental analysis of the predictive relation- 
ships between indicators and outcomes. Findings from the logistic regression and random forest analyses were 
consistent. The report presents results from the logistic regressions because this is the more common method 
and is more easily interpreted. (The random forest methods and findings are presented in appendix C.) 


Including middle school or high school indicators in the analysis improved the predictive accuracy for each 
outcome compared with models that included student background characteristics alone. Student background 
characteristics accurately predicted the readiness outcome (ACT score) and success outcomes (college enrollment 
and persistence) for approximately two-thirds to three-fourths of students, depending on the outcome. Including 
the set of high school indicators increased the predictive accuracy by 9 percentage points for each outcome, 
while including the set of middle school indicators increased the predictive accuracy by 4—8 percentage points. 


5. Students who were present more than 95 percent of days enrolled had a 12 percentage point higher attainment of the readiness 
outcome, a major difference, than students who were chronically absent (missed 10 percent or more days enrolled), while students 
who were present 91-95 percent of days enrolled did not demonstrate a major difference in attainment (8 percentage point differ- 
ence) compared with students who were chronically absent. 

6. The regression analyses addressing research questions 2 and 3 excluded students lacking ACT records. Appendix A summarizes a sensi- 
tivity analysis examining whether estimates changed if students lacking ACT records were included in the analysis. 


REL 2021-091 7 


Middle school indicators accurately predicted the readiness outcome (ACT score) for 82 percent of 
students who took the ACT exam; high school indicators did so for 83 percent of students 


The sets of middle school and high school indicators, along with student background characteristics, accurately 
predicted student attainment of the postsecondary readiness outcome (ACT score) for large majorities of stu- 
dents (figure 3; see table B6 in appendix B). Compared with predicting outcomes using only student background 
characteristics, the addition of middle school or high school indicators in the analysis notably increased the per- 
centage of students who were accurately predicted to not attain the outcomes (defined as true negatives) and 
decreased the percentage of students who were inaccurately predicted to attain the outcomes (defined as false 
positives). The high school indicators also performed better in this respect than the middle school indicators (see 
table B6). Greater accuracy in predicting which students might not attain outcomes is desirable in order to target 
supports to students who are mostly likely to need them. 


Middle school indicators accurately predicted enrollment success for 72 percent of students; high 
school indicators did so for 76 percent of students 


The set of middle school indicators plus student background characteristics accurately predicted student attain- 
ment of the enrollment success outcome for 72 percent of students. The set of high school indicators plus student 
background characteristics accurately predicted the enrollment success outcome for 76 percent of students. 
Student background characteristics alone accurately predicted the enrollment success outcome for 67 percent of 
students (figure 4; see table B6 in appendix B). 


Middle school indicators accurately predicted persistence success for 70 percent of students; high 
school indicators did so for 75 percent of students 


The set of middle school indicators plus student background characteristics accurately predicted student attain- 
ment of the persistence success outcome for 70 percent of students. The set of high school indicators plus student 
background characteristics accurately predicted the persistence success outcome for 75 percent of students. 


Figure 3. The middle school and high school indicators predicted attainment of the postsecondary readiness 
outcome (ACT score of 90 or higher) more accurately than background characteristics alone among students 
in the 2008/09 and 2009/10 grade 6 cohorts, 2008/09-2017/18 


Percent of students 
100 


80 


60 


40 


20 


Background Middle school indicators + High school indicators + 
characteristics background characteristics background characteristics 


Note: n = 11,441 (30 percent testing sample for students who took the ACT). 


Source: Authors’ analysis of data for 2008/09-2017/18 from the Arkansas Department of Education, Arkansas Division of Higher Education, National 
Student Clearinghouse, and National Center for Education Statistics Common Core of Data (U.S. Department of Education, n.d.). 
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Student background characteristics alone accurately predicted the persistence success outcome for 66 percent of 
students (figure 4; see table B6 in appendix B). 


Predictive strength of middle school and high school indicators 


This section presents findings on the predictive strength of middle school and high school indicators associated 
with students’ progression toward postsecondary readiness and success (see table A3 in appendix A for the per- 
centages of students exhibiting each indicator).’ Appendix B provides supporting tables. 


Math and English language arts proficiency, regular school attendance, no suspensions, and no 
expulsions in middle school were major predictors of the postsecondary readiness and success outcomes 


Middle school academic indicators were major predictors of the readiness outcome (ACT score), while several 
middle school academic, attendance, and discipline indicators were major predictors of the success outcome 
(college enrollment and persistence). Specifically, English language arts and math proficiency were major pre- 
dictors of all three outcomes examined in the study, whereas no suspensions, no expulsions, and regular school 
attendence were major predictors of the success outcomes (college enrollment and persistence).® 


Figure 4. The middle school and high school indicators predicted attainment of the success outcomes (college 
enrollment and persistence) more accurately than background characteristics alone among students in the 
2008/09 and 2009/10 grade 6 cohorts, 2008/09-2017/18 


@ Enrollment? @ Persistence? 
Percent of students 
100 - 


30 ——_____— — 


60 —— 


40 


20 —j 


Background Middle school indicators + High school indicators + 
characteristics background characteristics background characteristics 


Note: n = 19,103 (30 percent testing sample for all students) 
a. Enrolled in college for at least one term, regardless of the degree or certificate being pursued or attained, within eight years of beginning grade 6. 
b. Enrolled in college for more than one term within eight years of beginning grade 6. 


Source: Authors’ analysis of data for 2008/09-2017/18 from the Arkansas Department of Education, Arkansas Division of Higher Education, National 
Student Clearinghouse, and National Center for Education Statistics Common Core of Data (U.S. Department of Education, n.d.). 


7. The study defined predictive strength as the correlation between an individual indicator and an outcome measure, based on a regres- 
sion model controlling for other indicators and student background characteristics. The study defined major predictors as indicators 
with a statistically significant correlation with an outcome and also predicting at least a 10 percentage point change in the probability 
of a student attaining an outcome. 

8. Although the “never expelled” indicator is associated with a substantial estimated percentage point change in the probability of attain- 
ing the readiness outcome (ACT score), this association is not statistically significant, and thus the indicator is not discussed as a major 
predictor of the readiness outcome. 
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Math, science, and English language arts proficiency in middle school were major predictors of the readiness 
outcome. Students who demonstrated proficiency in math, science, or English language arts on state assessments 
during middle school were 24—28 percentage points more likely to score 19 or higher on the ACT exam than were 
other students (figure 5; see table B7 in appendix B). 


No expulsions, no suspensions, regular school attendance, and math and English language arts proficiency in 
middle school were major predictors of success outcomes (college enrollment and persistence). Students who were 
never expelled were 29 percentage points more likely than those who were ever expelled to enroll in college for 
at least one term within eight years of beginning grade 6 (figure 6; see table B8 in appendix B). Students who 
attended more than 95 percent of days enrolled, demonstrated proficiency in English language arts in grade 8, 
attended 91—95 percent of days enrolled, were never suspended, or demonstrated proficiency in math in grade 8 
were 10-19 percentage points more likely than students without these characteristics to enroll in college. 


Students who were never expelled were 27 percentage points more likely than those who were ever expelled to 
persist in college for more than one term within eight years of beginning grade 6, and students who attended 
more than 95 percent of days enrolled were 20 percentage points more likely than students who attended less 
often (see figure 6 and B8 in appendix B). Students who demonstrated proficiency in English language arts in 
grade 8, attended 91-95 percent of days enrolled, demonstrated proficiency in math in grade 8, or were never 
suspended were 11-13 percentage points more likely than others to attain this outcome. 


Figure 5. Three middle school indicators were major predictors of the readiness outcome (ACT score of 19 or 
higher) among students in the 2008/09 and 2009/10 grade 6 cohorts, 2008/09-2017/18 


Percentage point change in likelihood of scoring 19 or higher on the ACT 


40 
30 
27 
20 j 
17 
10) ~ ~ 7 ES a k= 
-1 2 
-10 
Proficient Proficient in Proficient in Never Never Present Present 
in math*™* science’ English expelled suspended” morethan 91-95 percent 
language 95 percent of days 
arts’ of days enrolled 
enrolled 


*** is significant at p < .001. 


Note: n = 11,441 (30 percent testing sample for students who took the ACT). Estimates are based on the marginal effects presented in table B7 in appen- 
dix B. An indicator is considered a major predictor if it has a statistically significant association with an outcome and predicts at least a 10 percentage 
point change in the probability of a student attaining an outcome, after other variables in the model are controlled for. 


Source: Authors’ analysis of data for 2008/09-2017/18 from the Arkansas Department of Education, Arkansas Division of Higher Education, National 
Student Clearinghouse, and National Center for Education Statistics Common Core of Data (U.S. Department of Education, n.d.). 
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Figure 6. Six middle school indicators were major predictors of the success outcomes (college enrollment and 
persistence) among students in the 2008/09 and 2009/10 grade 6 cohorts, 2008/09-2017/18 


@ Enrollment? ™ Persistence? 


Percentage point change in likelihood of attaining postsecondary success outcome 
40 


Phin 


Never Present Proficient in Present Never Proficient — Proficient in 
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Note: n = 19,103 (30 percent testing sample for all students). Estimates are based on the marginal effects presented in table B8 in appendix B. An indica- 
tor is considered a major predictor if it has a statistically significant association with an outcome and predicts at least a 10-percentage point change in 
the predicted probability of a student attaining an outcome, after other variables in the model are controlled for. 


a. Enrolled in college for at least one term, regardless of the degree or certificate being pursued or attained, within eight years of beginning grade 6. 
b. Enrolled in college for more than one term within eight years of beginning grade 6. 


Source: Authors’ analysis of data for 2008/09-2017/18 from the Arkansas Department of Education, Arkansas Division of Higher Education, National 
Student Clearinghouse, and National Center for Education Statistics Common Core of Data (U.S. Department of Education, n.d.). 


Grade point average, enrollment in an advanced course, no expulsions, and present more than 
90 percent of days enrolled (not chronically absent) in high school were major predictors of two or 
more of the postsecondary readiness and success outcomes 


High school academic indicators were major predictors of the readiness outcome (ACT score), while several high 
school academic, disciplinary, and attendance indicators were major predictors of the success outcomes (college 
enrollment and persistence). Specifically, having a higher grade point average was a major predictor of all three 
outcomes examined in the study, and taking an advanced course, not being expelled, and attending school regu- 
larly were major predictors of the success outcomes (college enrollment and persistence). 


Science and math proficiency and grade point average in high school were major predictors of the readiness 
outcome. Students who demonstrated proficiency on the high school science (biology) assessment were 33 per- 
centage points more likely than other students to score 19 or higher on the ACT exam (figure 7; see table B9 in 
appendix B). Students who earned a grade point average of 2.8 or higher were 17 percentage points more likely 
than other students to attain this postsecondary readiness outcome, and students who demonstrated proficiency 
at least once on the high school algebra or geometry assessments were 14 percentage points more likely. 


Major high school predictors of the success outcomes (college enrollment and persistence) were enrollment in an 
advanced course, no expulsions, a higher grade point average, and present more than 90 percent of days enrolled. 
Students who took at least one advanced course were 24 percentage points more likely than other students to 
enroll in college for at least one term within eight years of beginning grade 6 and 23 percentage points more likely 
to persist by enrolling in more than one term (figure 8; see table B10 in appendix B). Students who were never 
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Figure 7. Three high school indicators were major predictors of the readiness outcome (ACT score of 19 or 
higher) among students in the 2008/09 and 2009/10 grade 6 cohorts, 2008/09-2017/18 
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Source: Authors’ analysis of data for 2008/09-2017/18 from the Arkansas Department of Education, Arkansas Division of Higher Education, National 
Student Clearinghouse, and National Center for Education Statistics Common Core of Data (U.S. Department of Education, n.d.). 


Figure 8. Five high school indicators were major predictors of the success outcomes (college enrollment and 
persistence) among students in the 2008/09 and 2009/10 grade 6 cohorts, 2008/09-2017/18 
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cator is considered a major predictor if it has a statistically significant association with an outcome and predicts at least a 10 percentage point change in 
the probability of a student attaining an outcome, after other variables in the model are controlled for. 


a. Enrolled in college for at least one term, regardless of the degree or certificate being pursued or attained, within eight years of beginning grade 6. 
b. Enrolled in college for more than one term within eight years of beginning grade 6. 


Source: Authors’ analysis of data for 2008/09-2017/18 from the Arkansas Department of Education, Arkansas Division of Higher Education, National 
Student Clearinghouse, and National Center for Education Statistics Common Core of Data (U.S. Department of Education, n.d.). 
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expelled were 17 percentage points more likely than other students to enroll in college and persist. Students who 
earned a grade point average of 2.8 or higher were 16 percentage points more likely than other students to enroll 
in college and 17 percentage points more likely to persist. Students who attended more than 95 percent of days 
enrolled or 91-95 percent of days enrolled (were not chronically absent) were 11-15 percentage points more likely 
to enroll and persist than students who were chronically absent. Enrollment in a community service course was 
the one high school indicator investigated in the models of college enrollment and persistence that was not sig- 
nificantly correlated with either of these two success outcomes. 


Implications 


The study aimed to provide the Arkansas Department of Education and other education agencies in the state 
with new evidence on the use of early indicators of postsecondary readiness to identify students who are not on 
track for postsecondary readiness and success and who might benefit from intervention or supports. The study 
findings suggest implications about how awareness and use of postsecondary readiness indicators from middle 
school and high school could benefit education agencies and their students. 


More than half of Arkansas students might need intervention or supports to prepare them for 
postsecondary readiness and success 


Findings from this study suggest that large numbers of students are at risk of not meeting the Arkansas Depart- 
ment of Education’s goal for college readiness, as measured by an ACT score of 19 or higher. Although there are 
other ways to measure readiness, Arkansas values the ACT score as evidence that students have gained a basic 
level of skills and knowledge during K-12 schooling that readies them for college. Further, findings from this study 
reveal that many students did not attain the success outcomes of college enrollment and persistence. Arkansas 
students could benefit from further intervention or supports to prepare them for opportunities after high school. 
State and local education leaders and policymakers could use available data and validated early warning indica- 
tors to target intervention and resources to schools and students in greatest need. 


Indicators aligned with the Arkansas Every Student Succeeds Act plan can identify students in need of 
support 


The indicators examined in the study demonstrated accuracy in predicting outcomes for 70-83 percent of stu- 
dents when combined with information on student background characteristics. These types of indicators could 
be applied to identify individual students and groups of students, as early as the middle grades, who could benefit 
from intervention to support students’ progress and development and as a starting point for raising awareness 
and prompting action. For example, the Arkansas Department of Education could summarize, on publicly avail- 
able data dashboards and student report cards, student progress or status on key middle school and high school 
indicators and their attainment of college readiness outcomes. 


In discussing this study with the study team, leaders from the Arkansas Department of Education have expressed 
their intention to use the report’s findings and featured indicators to engage guidance counselors, teachers, and 
administrators at both secondary and elementary school levels and higher education leaders. When using indica- 
tors such as those in the study, educators need to consider the complexity of the associations, especially the way 
that earlier educational and environmental experiences can affect the levels of middle school and high school 
indicators as well as later outcomes. Some indicators, such as those measuring academic performance, might 
relate directly to postsecondary outcomes, while other indicators, such as disciplinary actions, might be symp- 
tomatic of other student needs or challenges. 
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Student attendance, grade point average, course rigor, proficiency, and disciplinary actions should be 
priorities for identifying risk and supporting students in middle school and high school 


The findings suggest the value of exploring further whether the examined indicators can not only predict post- 
secondary readiness and success but can also suggest ways to improve postsecondary outcomes by focusing on 
the behaviors represented by the indicators. For example, promoting regular attendance and preventing chronic 
absenteeism are a promising area for further investigation and possible intervention, based on this study’s find- 
ings as well as prior literature (Allensworth & Easton, 2005; Finn & Zimmer, 2012). Further study might point to 
ways in which issues as wide ranging as transportation, housing stability, nutrition, health challenges, stresses in 
the household that compete with regular attendance at school, or a school’s success in establishing a safe and 
inviting learning environment could be points of engagement for evidence-based intervention. 


The findings highlight the importance of high school grade point average and enrollment in advanced courses for 
postsecondary readiness and success, consistent with recent research (Allensworth and Clark, 2020). Grade point 
average encompasses multiple dimensions of academic engagement and performance and provides a formative 
signal—albeit one that could carry biases—that communicates meaningful information each semester on a student’s 
postsecondary readiness (Allensworth and Clark, 2020; Gayles, 2012). Similarly, taking advanced courses represents 
aspects of academic motivation and preparation that are associated with future success (Singh et al., 2002). 


This study also points to the importance of developing proficiency in English language arts, math, and science and 
thus to the value of intervention supports for students who are not mastering learning objectives in these core 
subject areas. School systems’ use of multiple tiers of monitoring and support—that become increasingly target- 
ed and intensive for identified groups of students—has proven effective (Utley & Obiakor, 2015). 


Finally, the study points to the risks of student suspension and expulsion. Monitoring and supporting a support- 
ive social and emotional environment in schools can reduce student behavioral problems and the frequency of 
suspensions and expulsions (Kopershock et al., 2016). Further, when disciplinary problems arise, nonpunitive 
alternatives to suspension and expulsion, including restorative justice practices and other relationship-centered 
approaches, show promise for addressing harm and collaboratively solving problems (Fronius et al., 2019). These 
approaches could reduce the need for student suspension or expulsion. 
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